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Cost  Cumulant-Based  Control  for  a  Class  of  Linear  Quadratic  Tracking  Problems 


Khanh  D.  Pham 
Space  Vehicles  Directorate 
Air  Force  Research  Laboratory 
Kirtland  AFB,  NM  87117  U.S.A. 


Abstract — The  topic  of  cost  cumulant  control  is  currently 
receiving  substantial  research  from  the  theoretical  community 
oriented  toward  stochastic  control  theory.  For  instance,  the 
present  paper  extends  the  application  of  cost  cumulant  con¬ 
troller  design  to  control  of  a  wide  class  of  linear  quadratic 
tracking  systems.  It  is  shown  that  the  tracking  problem  can  be 
solved  in  two  parts:  a  feedback  fc-cost-cumulant  (kCC)  control 
whose  optimization  criterion  representing  a  linear  combination 
of  Unite  k  cumulant  indices  of  a  finite  horizon  integral  quadratic 
cost  associated  to  a  linear  tracking  stochastic  system  is  deter¬ 
mined  by  a  set  of  Riccati-type  differential  equations  and  a 
set  of  time-dependent  tracking  variables  is  found  by  solving  an 
auxiliary  set  of  differential  equations  (incorporating  the  desired 
trajectory)  backward  from  a  stable  final  time. 

I.  Preliminaries 

An  interesting  extension  of  the  cost-cumulant  control  the¬ 
ory  [ l]-[5]  when  both  perfect  and  noisy  state  measurements 
are  available,  is  to  consider  following  a  specified  output 
trajectory  as  closely  as  possible  in  the  sense  of  cost-cumulant 
control  objective.  Some  motivations  for  this  theoretical  devel¬ 
opment  are  found  in  the  altitude  control  of  a  terrain-following 
aircraft  where  there  is  knowledge  of  the  future  terrain;  and  in 
tactical  and  combat  situations  wherein  a  vehicle  with  the  goal 
seeking  nature  initially  decides  on  an  appropriate  destination 
and  then  moves  in  an  optimal  fashion  toward  that  destination. 
Consider  a  linear  stochastic  tracking  system  governed  by 

dx{t)  =  (A(t)x(t)  +  B(t)u{t))dt  +  G(t)dw(t),  x(to )  (1) 
y(t)  =  C(t)x(t)  (2) 

where  the  coefficients  A  £  C([f0, tf]\ R"x"),  B  £ 

C([i0,t/];Rnx,n),  C  £  C([f0A/];Rrxn),  and  G  £ 

C([fo, tf]; Rraxp).  The  system  noise  w(t)  £  Rp  is  the 
p-dimensional  stationary  Wiener  process  starting  from  to, 
independent  of  x(to)  =  Xq,  and  defined  on  a  complete 
probability  space  over  [to,tf]  with  the  correlation 

E  {[w(t)  —  w(£)]['u;(t)  —  ,U’(C)]T}  =  W\t  —  £| ,  W  >  0  . 

The  control  input  u  £  L^t  (Q;  C([to,  tf];  Mm))  the  subset 
of  Hilbert  space  of  Rm- valued  square-integrable  process 
on  [to,tf]  that  are  adapted  to  the  er-field  Tt  generated  by 
w(t)  to  the  specified  system  model  is  selected  so  that  the 
resulting  output  y  e  L%t  (Q;  C([to,  tf]; Kr))  best  matches 
the  desired  output  z  £  L2([f0,  tf];  Rr)  in  the  cost  cumulant 
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optimization  criterion  which  will  be  clear  shortly.  Associated 
with  the  initial  condition  ( to,Xo;u )  £  x  Rn  x 

Lj7t(Q;C([to,tf];Mm))  is  a  traditional  finite-horizon  IQF 
random  cost  J  :  [to,tf]  x  x  L^t  (Cl;  C([to,  tf];  R171))  >— > 
R+  such  that 

J(t0,x0;u )  =  [z(tf)  -  y{tf)]TQf  [z(tf)  -  y(tf)\  (3) 

+  [  {[z{T)—y(T)]TQ{T)  [z(t) -2/(t)]  +  wt(t)R(t)w(t)}g1t 

J  to 

in  which  the  terminal  penalty  error  weighting  Qf  £  Rrxr, 
the  error  weighting  Q  £  C([fo,  t /] ;  Krxr),  and  the  control 
input  weighting  R  £  C([to,  f/];  Kmxm)  are  symmetric  and 
positive  semidefinite  with  R(t)  invertible. 

In  the  perfect-state  measurement  case,  the  initial  state 
is  assumed  to  be  known  exactly,  and  the  control  input  is 
generated  by  a  closed-loop  control  policy  of  interest  7  : 
[to,  tf]  x  L%  (fi;  C([t0,  tf]\ K"))  ^  L\ (U;  C([t0,  tf];  Km)), 
according  to  the  control  law 

u(t)  =  7 (t,  x(t))  =  K(t)x(t)  +  uext(t) ,  (4) 

where  uext  £  C([to,  f/];  Mm)  is  an  external  signal  and  K  £ 
C([to,  tf]',  Kmxn)  is  an  admissible  feedback  gain  in  a  sense 
to  be  specified  later.  Hence,  for  the  given  initial  condition 
(t o,Xo)  G  [to ,tf]  x  R”  and  subject  to  the  control  policy  (4), 
the  dynamics  of  the  tracking  problem  are  governed  by 

dx{t)  =  [A(t)  +  B(t)K(t)]x{t)dt  +  B(t)uext(t)dt 

+  G(t)dw(t) ,  x(t0)  =  x0 ,  (5) 

y(t)  =  C(t)x(t) ,  (6) 

and  the  IQF  random  cost 

J(t0,x0;K,uext)  =  [ z(tf )  -  y{tf)]T  Qf  [. z(tf )  -  y{tf)] 

+  [  {  Ht)  -  y(T)f  Q(t )  iz(T)  -  y(T)}  (7) 

7  to  k 

+  [K(t)x{t)  +  uext{T)}TR(T)[K{T)x{r)  +  uext(r)]^dT. 

It  is  now  ready  to  generate  some  cost  cumulants  for  the 
finite-horizon  tracking  problem.  These  cost  statistics  are  sub¬ 
sequently  utilized  in  defining  the  performance  index  arisen 
in  state-feedback  fcCC  control  over  a  finite  horizon  opti¬ 
mization.  In  general,  it  is  suggested  that  the  initial  condition 
(t q,Xo)  should  be  replaced  by  any  arbitrary  pair  (a,xa). 
Then,  for  the  given  external  signal  uext  and  admissible 
feedback  gain  I\ ,  the  cost  functional  (7)  is  seen  as  the  “cost- 
to-go”,  J  (a,  xa).  The  moment-generating  function  of  the 


vector-valued  random  process  (5)  is  given  by  the  definition 

<p(o:,xa;6)  =  E  {exp(9J  (a,xa))}  ,  (8) 

where  the  scalar  9  £  R+  is  a  small  parameter.  Thus,  the 
cumulant-generating  function  immediately  follows 

ip(a,xa-,0)  =  \n{ip(a,xa-,0)}  ,  (9) 

in  which  ln{-}  denotes  the  natural  logarithmic  transformation 
of  an  enclosed  entity. 

Theorem  1:  For  all  a  £  [io ,  £/]  and  the 
small  parameter  9  £  R+,  define  ip(a,xa;9 )  = 
g  {a,  9)  exp  {x^Y(a,  0)xa  +  2x^ri(a,  9) }  and  v{a,9)  = 
ln{g(a,  0)}.  Then,  the  cost  cumulant-generating  function 
can  be  expressed  as  follows 

(a,  xa\  9)  =  x^Y(a,  9)xa  +  2 x^r](a,  9)  +  v  (a,  9)  (10) 

where  T(a,  9),  r](a,9),  and  v(a,9)  solve  the  backward-in- 
time  differential  equations 

=  -[A{a)  +  B{a)K{a)]TY{a19)  (11) 

-T  (a,9)[A(a)  +  B{a)K(a)\ 

-  2T(a,  9)G(a)WGT(a)Y(a,  9) 

—  9CT  {a)Q{a)C{a)  —  9KT  (a)R(a)K(a) , 

=  —  \A(a)  +  B(a)K(a)}Tr](a,  9)  (12) 

-  T (a,9)B(a)uext(a) 

—  9KT (a)R(a)uext(a)  +  9CT (a)Q(a)z(a) , 

=  -Tr{T (a,9)G{a)WGT  {a)}  (13) 

-  2rj T(a,  9)B(a)uext{a) 

—  9u^xt{a)R(a)uext{a)  —  9zT(a)Q(a)z(a) 

with  the  terminal  conditions  T (tf,9)  =  9CT (tf)Q 
V  (tf,  9)  =  9CT(tf)Qfz(tf),  v  ( tf ,  9)  =  0  zT  {t  f)Q  f  z(t  f) . 
Proof.  For  any  9  given,  let  w  {a,  xa\  9)  =  exp  {9  J  (a,  a:,*)}, 
then  the  moment-generating  function  becomes 
< p  (a,  xa;  9)  —  E  {zu  (a,  xa;  0)}  with  the  time  derivative  of 

-^ip(a,xa;9)  =  -6^x^[CT(a)Q(a)C(a) 

+  KT(a)R(a)K(a)\ xa  +  2x^[KT  {a)R{a)uext{a) 

-  CT(a)Q(a)z(a )]  +  uJxt(a)R(a)uext(a) 

+  zT(a)Q(a)z(a)Jip(a,xa;  9) .  (14) 
Using  the  standard  Ito’s  formula,  it  yields 

dip  (a,  xa\  9)  =  E  {dvj  (ct,  xa ;  0)}  , 

=  E^wa  (a,  xa ;  9)  da  +  vjXa  (a,  xa;  9)  dxa 

+  ^Tr  {zuXaXa(a,xa;  9)G(a)WGT(a)j  daj  , 

=  ipXa  (a,  xa\  9)  [A(a)  +  B(a)K(a)\  xc ,da 
+  ipa  (a,  xa ;  9)  da  +  ipXa  (a,  xa\  9)  B(a)uext(a)da 

+  ^Tr  {ipXaXa  (a,  xa;  9)  G  (a)  WGT  (a)}  da, 


da 


T  (a,  9) 


—ij  (a,  9) 
da 


dZ,vta-e) 


which  under  the  definition  ip(a,xa\9)  = 
g(a,9)  exp  {x^T(a,9)xa  +  2x^ij(a,9)}  and  the  partial 
derivatives 


ipa  ( a,xa;  9)  = 

+  x^T<a'  e)x" +  ") 


ip(a,xa;9 ) 


Txa  (ct,  xa]  9)  = 

{x^  [T(q,  9)  +  YT(a,  9)]  +  2 if(a,9)}ip(a,xa;9) , 
T>xaxa  ( a,xa\9 )  =  [T(a,0)  +  T1  (a,  0)]  ip(a,xa;9) 

+  [T (a,  9)  +rT(a,  9)]  xaxTa  [T(a,  9)  +TT(a,  0)]  ip  (a,  xa-,  9) 


leads  to 


<P  («,  xa\  9)  =  do,^a'n?  V  (a,  xa;  9) 
da  g(a,  u) 

(a,9)xa  +  2x^-^-r](a,9) 
da  da 


ip(a,xa-,9) 


+  x t  [A(a)  +  B(a)K(a)}1  T(a,  9)xaip  (a,  xa\  9) 

+  x^T(a,  9)  [A{a)  +  B(a)I<(a)\  xaip  (a,  xa\  9) 

+  2x^  [ A(a )  +  B(a)K (a)]T  rj(a,  9)ip  (a,  xa ;  9) 

+  2x^T(a,  9)B(a)uext(a)ip  (a,  xa ;  9) 

+  2r]T (a,  9)B(a)uext(a)ip  (a,  xa;  9) 

+  Tr  {T(a,  9)G{a)WGT(a)}  ip  (a,  xa\ 9) 

+  2a;^T(a,  0)G(a)lUGT(o;)T(a,  9)xaip  (a,  xa;  9)  .  (15) 


Replacing  (14)  into  (15)  and  having  both  linear  and  quadratic 
terms  independent  of  xa,  it  requires  that 

9)  =  -\A(a)  +  B(a)K(a)]T  T(a,  9) 
da 

-T(a,9)[A(a)  +  B(a)K(a)} 

-  2T(a,  9)G(a)WGT(a)T(a,  9) 

—  9Ct  (a)Q(a)C(a)  —  9KT(a)R(a)K(a) , 

-—r]  ( a ,  9)  =  —\A{a)  +  B(a)K(a)]T g{a,  9) 
da 

-  T (a,9)B(a)uext.(a) 

—  9I\t (a)R(a)uext(a)  +  9CT (a)Q(a)z(a) , 

■^—v  (a,  9)  =  — Tr  {T(a,  9)G  (a)  1UGT  (a) } 
da 

-  2r]T(a,  9)B(a)uext(a ) 

—  9u^xt{a)R(a)uext{a)  —  9zT(a)Q(a)z(a) . 

At  the  final  time  a  =  tf,  it  follows  that  <p(tf,x{tf)\9)  = 
e(tf,  9)  exp  {xT(tf)T(tf,  9)x{tf)  +  2xT{tf)r1{tf,9)}  = 

E  {exp  {9[z(tf)  —  y(tf)]TQf[z(tf)  —  y(tf)\}}  which  in 
turn  yields  the  terminal  conditions  as 

T  {tf,9)  =  9CT{tf)QfC{tf), 

V (tf,9)  =  —9CT(tf)Qfz(tf) , 
e(tf,9)  =  exp{9zT(tf)Qfz{tf)}  , 
v(tf,9)  =  9zT (tf)Qfz(tf) . 


Remark.  The  expression  for  cost  cumulants  (10)  in  the 
tracking  problem  indicates  that  additional  second  and  third 
affine  terms  are  taking  into  account  of  dynamics  mismatched 
in  their  trajectory-governing  equations. 

By  definition,  cost  cumulants  for  the  tracking  problem  can 
be  generated  by  employing  the  MacLaurin  series  expansion 
for  the  cumulant-generating  function 


(a,  xa;  9)  =  ^2  «*(«.  xa)  — 


(16) 


i=l 


i=l 


6=0 


in  which  m(a,x a)  are  called  cost  cumulants.  Furthermore, 
the  series  coefficients  of  the  expansion  is  computed  by  using 
the  result  (10) 


gi 

—rp{a,xa;9) 


=  xl 


6=0 


2xl 


dei 


6=0 


3=0 


gi 

d¥ 


’{a,  9) 


(17) 


6=0 


In  view  of  the  results  (16)  and  (17),  we  may  obtain  cost 
cumulants  for  the  tracking  problem  as  described  below 


T  dl 


Ki(a,xa)  =  xa  (a,  9) 


T  dl 


6=0 


xa+2xa  —^(a.,6) 


+  w  >'(«.«) 


6=0 

(18) 


6=0 


6=0 


for  any  finite  1  <  i  <  oo.  For  notational  convenience,  denote 

(a, 9)  ,  D(a,i )  =  -§^i i(a,9) 

0=0 

D(a,i)  =  ^grv(a,9)  .  Then,  we  would  like  to  state  the 

following  theorem. 

Theorem  2:  (Cost  Cumulants  in  Tracking  Problems) 

The  system  dynamics  governed  by  the  linear  stochastic 
differential  equations  (5)-(6)  attempt  to  track  the  prescribed 
signal  z(t)  with  the  finite-horizon  IQF  cost  (7).  For  k  £  Z+ 
fixed,  the  fcth  cost  cumulant  in  the  tracking  problem  is  given 

Kk(to,Xo',K,uext)  =  xxH(t0,k)x0 


2xxD(t0,k)  +  D(t0,k) ,  (19) 


in  which  the  building  variables  {H(a,  *)}f=1, 
and  {D(a,i) }*=1  evaluated  at  a  =  to  satisfy  the  differential 
equations  (with  the  dependence  of  H(a,i),  and 

D(a,i )  upon  uext  and  K  suppressed) 

—  H(a,  1)  =  —  [A(a )  +  B(a)K(a)]T  H(a,  1) 

da 

—  H(a,  1)  [A(a)  +  B(a)K(a)] 

—  CT(a)Q(a)C(a)  -  KT (a)R(a)K(a) ,  (20) 

—H{a,  i)  =  —  [A(a)  +  B(a)K(a))T  H(a,  i) 

—  H{a ,  T)  [.4(a)  +  B{a)K{a)) 

0*1 

—  V  or  -M  H(a,j)G{a)WGT{a)H(a,i-j)  (21) 


together  with 

4~D(a,  1)  =  -  \A(a)  +  B(a)K(a)}T  D{a ,  1)  (22) 

da 

-  H(a,T)B{a)uext{a) 

-  KT  (a)R(a)uext(a)  +  CT (a)Q(a)z(a) , 

—D(a,  i)  =  —  [A(a )  +  B(a)K (a)]T  D(a,  i ) 
da 

-  H(a,i)B(a)uext(a),  2  <i<k,  (23) 

and 

~^D(a,  1)  =  -Tr  {H(a,  1  )G(a)WGT(a)}  (24) 

-  2DT(a,  1  )B(a)uext(a) 

-  uJxt(a)R(a)uext{ai)  -  zT(a)Q(a)z(a) , 
~^D(a,i)  =  -Tr  {H (a,  i)G(a)WGT (a)} 

-  2 DT (a,i)B(a)uext(a),  2  <  i  <  k  (25) 

where  the  terminal  conditions  H(tf,  1)  =  CT(tf)QfC(tf), 
)  =  0  for  2  <  i  <  k;  D(tf,  1)  =  —CT(tf)Qfz(tf), 
D(tf,i)  =  0  for  2  <i<k  and  D{tf,l)  =  zT(tf)Qfz(tf), 
D(tf,  i)  =  0  for  2  <  i  <  k. 

Proof.  Note  that  the  equations  (20),  (22)  and  (24)  satisfied 
by  H(a,  1),  D(a,  1),  and  D(a,  1)  can  be  obtained  by  taking 
the  derivative  with  respect  to  9  of  the  equations  ( 1 1)-(  1 3) 

ir  =  " [A(a)  +  teT^e) 

-^T (a,  9)  [. A(a)  +  B(a)K(a )] 
-2  |  J^T (a,  9)  j  G{a)WGT{a)Y{a1 9) 

— 2T  (a,  9)  G  (a)  WGT  (a)  |  J^Y(a,0)| 

— CT(a)Q(a)C(a)  —  KT (a)R(a)K(a) , 

^  }  =  “  t A +  B(a)K(a)lT  9) 

(a,  9)B(a)uext(a) 

— KT (a)R(a)uext(a)  +  CT (a)Q(a)z(a) , 

1{WV^}  =  -T,  {^(a,e)G(a)WG^)} 

d 

-2 —if  {a,  9)B(a)uext{a) 

- uxxt(a)R(a)uext{a )  -  zT (a)Q(a)z(a) , 

wherein  terminal  conditions  JgT(f/,0)  =  CT (t f)Q fC(t /), 
Mjl(tf,9)  =  —CT(tf)Qfz(tf),  and  ^ v(tf,9 )  = 
zT(tf)Qfz(tf).  It  is  important  to  see  that  when  0  =  0  the 
equation  (11)  becomes 

0)  =  - \A(a)  +  B{a)I<{a)}T T(a,  0) 
da 

—  Y(a,  0)[A(a)  +  B(a)K  (a)] 

-  2T(a,  0)G{a)WGT{a)T{a,  0) ,  T (tf,  0)  =  0 . 


Because  the  closed-loop  matrix  A(a)  +  B(a)K(a)  is  as¬ 
sumed  stable  for  all  a  G  [to,tf],  it  is  then  deduced  that 
T (a,  0)  =  0.  Using  this  result  together  with  the  definitions  of 

H(a,l)  =  MT(a’6')L=o’  D(a,l)  =  mrl(a’e)\e= o’  and 

D(a,  1)  =  J^v(a,  9)  |e _Q,  the  first  cost  cumulant  is  found 
Ki(t0,  x0;  K)  =  xlH{t0,  l)x0  +  2xq -D(io,  1)  +  D(t0, 1) , 

where  the  solutions  H(a,  1),  D(a,  1)  and  D(a,  1)  satisfy 
the  backward-in-time  differential  equations 

—  H(a,  1)  =  —  L4(a)  +  B(a)K(a)]T  H(a,  1) 
da 

-  H(a,  1)  [A(a)  +  B(a)K(a)\ 

—  CT (a)Q(a)C(a)  —  KT(a)R(a)K(a) , 

4-D(a,  1)  =  -  \A(a)  +  B{a)K{a)]T  D{a,  1) 
da 

-  H(a,  l)B(a)uext(a) 

-  KT [a)R(a)uext{a)  +  CT (a)Q(a)z(a) , 

4-D(a,l)  =  — Tr \H(a,  l)G(a)W GT (a)\ 
da  k  J 

-  2 DT{a,  1  )B{a)uext{a) 

-  u^xt(a)R(a)uext(a)  -  zT (a)Q(a)z(a) , 


with  boundaries  H(tf,  1)  =  CT(tf)QfC(tf),  D(tf,  1)  = 
— CT(tf)Qfz(tf )  and  D(tf,  1)  =  zT(tf)Qfz(tf).  Repeat¬ 
edly,  taking  of  the  equations  ( 1 1)-(  1 3)  yield  the  corre¬ 
sponding  differential  equations 

1  {wiT{a'9)}  =  -  [A(a)  +  B{a)K{a)]T^T{a,9) 

[A(a)  +  B(a)K(a)} 
-2^T(a,  9)G(a)WGT(a)T(a ,  9) 
-4^T(a,  9)G{a)WGT{a)  ^T(a,  9) 
-2T(a,  9)G{a)WGT{a)  j^T(a,  9) , 
£  =  -[A(a)  +  S(a)/x(a)]TJ^r?(a,0) 

a2 

-^T(a,0)B(a)«ext(a) , 

1  {£*>(«'")}  =  -Tr{|,T(«,S)GWlr0-(«)} 

d2 

-2QQ2'nT(u,9)B(a)uext(a) , 


together  with  terminal  conditions  £,T(tf,9)  =  0, 

WtV(tf,9)  =  0,  and  -§ffzv(tf  ,9)  =  0.  Having  substituted 
H(a,  1)  =  ^T(a,0)|9=o,  H(a,  2)  =  ^T(a,6»)|  , 


^(«)2)  =  -M2V(®,9)  ,  D(a,  2)  =  ■§mv{a,9) 


0=0 


and 


0=0 


T(a,  0)|e=o  =  0  into  the  above  equations,  the  second  cost 
cumulant  can  be  obtained  as  follows 


K2(f0,  x0;  K)  =  XoH(t0, 2)x0  +  2xgD(t0, 2)  +  D(t0, 2) , 


in  which  the  solutions  H(a,  2),  D(a,  2)  and  D(a,  2)  evalu¬ 
ated  at  a  =  to,  are  solving  the  differential  equations 

~^H(a,  2)  =  -  [A(a)  +  B{a)K (a)]T  H{cx,  2) 

-H(a,2)  [A{a)  +  H(a)A"(a)] 

-4fJ(a,l)G(a)VUGT(a)7J(a,l) ,  H(tf,2)  =0, 

2)  =  -  L4(a)  +  B(a)K (a)]T  D{cx,  2) 
aa 

-H(a,2)B(a)uext{a) ,  D(tf,  2)  =  0 , 

-^D(a,  2)  =  — Tr  {fT(a,  2)G(a)fUGT(a) } 

-2DT(a,  2)B(a)uext(a) ,  D(tf,  2)  =  0 . 

By  successively  taking  derivatives  of  the  equations  ( 1 1)-(  1 3) 
with  respect  to  9  and  evaluating  the  results  at  9  =  0,  the  ith 
cost  cumulant  can  be  written  for  all  2  <  i  <  k 

Ki(t0,x0;K)  =  XoH(t0,i)x0  +  2xJ D(t0,i)  +  D(t0,i) , 

where  D(a,i )  and  D(a,i)  evaluated  at  a  =  to  are 

the  solutions  of  the  coupled  differential  equations 

—  H(a,  i)  =  —  \A(a)  +  B(a)K(a)]T  H(a,  i) 
da 

—  H(a,  i)  \A(a)  +  B(a)K(a)\ 

^  ^  2?  I 

-u-1  aH(aJ)G(a)WGT(a)H(a,i-j),H(tf,i)  =  0 
i=1.M*  3>- 

—  D(a,  i)  =  —  \A(a)  +  B(a)K(a)]T  D(a,  i) 
da 

—  H(a,i)B(a)uext(a) ,  D(tf,i)  =  0, 

^ D(a,i )  =  -Tr{ff(a,i)G(a)lUG'r(a)} 

-2 DT(a,i)B(a)uext(a) ,  D(tf,i)  =  0. 

Thus,  the  proof  is  now  complete. 

II.  Problem  Statements 

In  preparing  for  the  fcCC  control  statements  of  the 
tracking  problem,  let  fc-tuple  variables  Ti  and  V  be  de¬ 
fined  as  follows  7 -{(■)  =  (Hi(-), . . .  '£>{•)  = 

(©!(•),..., T)k(-)),  T>{-)  =  (2?1(-),...,2?fc(-))  for  each 

element  Tii  G  C1^,  tf\;  K"x")  of  H,  T>i  G  C1([f0,  </];  K") 
of  T>  and  T>t  G  C1([fo,  f/];  K)  of  I?  having  the  representations 

Hi(-)=H(;i),  T>i(-)  =  D(;  i),  ©<(■)  =  £>(-,*) . 

with  the  right  members  satisfying  the  dynamic  equations 
(20)-(25)  on  the  horizon  [f0,f/].  The  problem  formulation 
can  be  considerably  simplified  if  the  convenient  mappings 
are  introduced 

Ti  :  [to,tf]  x  (K™x")fc  x  Rmxn  ^  Knxn 

Gi  :  [to,tf]  x  {Wnxn)k  x  (K")fc  x  Kmxn  xT^K" 

Gi  :  [t0,tf]  x  (K"x”)fc  x  (K")fc  xPhR 


where  the  actions  are  given  by 

Tx  (a,H,K)  =  -  [A(a)  +  B(a)K(a)]TH1(a) 

-  Hi  (a)  [A(a)  +  B(a)K(a )] 

-  CT (a)Q(a)C(a)  —  KT(a)R(a)K(a ) , 
fi(a,H,K)  =  -  [A(a)  +  B(a)K(a)f  Hfia) 

-  Hfia)  [A(a)  +  B{a)K{a)} 

OjI 

-  -I cl  .vHj(a)G(a)WGT(a)Hi-j(a) , 

J=1JV  3)- 

Qi  (a,H,V,K,uex2j  =  -  \A(a)  +  B(a)K(a)]T  Vi{a) 

-  Hi(a)B(a)uext(a) 

-  Kt (a)R(a)uext(a)  +  CT(a)Q(a)z(a ) , 

Qi  (a,H,V,K,uex2j  =  -  \A(a)  +  B(a)K(a)]T  Vfia) 

-  Hi(a)B(a)uext(a) , 

Qi  (a,H,V,uex2)  =  -Tr{?G(a)G(a)FFGT(a)} 

-  2T>[ ( a)B{a)uext{a ) 

-  u2t(a)R(a)uext(a)  -  zT(a)Q(a)z(a ) , 

Qi  (a,H,V,uex2)  =  -Tr  {Hfia)G(a)WGT (a)} 

-  2 (a)B(a)uext(a) . 

Now  there  is  no  difficulty  to  establish  the  product  mappings 

Ti  x---xTk:[t0,tf)  x  (rx")ixlmxnH>  (K”x")fc 
QiY.---Y.Qk-.  [t0,  tf ]  y  (Rnxn)kY  (r)fcxlmx"xtmi->(r) 
QiY---YQk  :[t0,tf]  x  (K"x”)fe x  (r)fcxImh4Kfc 

along  with  the  corresponding  notations  T  =  T\  x  •  •  •  x  Tk, 
Q  =  Q i  x  •  •  •  x  Qk,  and  Q  =  Q\  x  •  •  •  x  Qk.  Thus,  the  dynamic 
equations  of  motion  (20)-(25)  can  be  rewritten  as 

-—H(a)  =  J:(ya,  H(a),K{a)),  H(tf)  =  Hr, 
da 

-^V(a)  =  Q^a,  H(a),V(a),K(a),uext(aj) ,  V(tf)  =  Vf, 
~^—'D(a)  =  Q  ( 'a,H(a),V(a),uext(a ))  ,  V(t.f)  =  Vf 

where  /c-tuple  values  Hf  =  (CT(tf)Q  fC(tf),  0,...,0), 
Vf  =  (-CT(tf)Qfz(tf),  0, . . . ,  0)  and  Vf  =  (0,...,  0).  ^ 
Note  that  the  product  system  uniquely  determines  H ,  V 
and  V  once  the  admissible  external  signal  uext  and  feedback 
gain  K  are  specified.  Hence,  they  are  considered  as  H  = 
H(-,  K),  V  =  V(-,K,uext)  and  V  =  V(-,K,uext).  The 
performance  index  in  ACC  control  problems  can  now  be 
formulated  in  uext  and  K. 

Definition  1:  (Performance  Index) 

Fix  k  £  Z+  and  the  sequence  /i  =  {/ij  >  0}*=1  with  fi i  >  0. 
Then  for  the  given  ( to .  xo  ) ,  the  performance  index 

ftk  ■  [to,  tf]  x  {Rnxn)k  x  ( Rn)k  x  Rk  i — ►  M+ 


in  finite-horizon  state-feedback  fcCC  control  for  the  tracking 
problem  is  defined  as 

<t>tk  (to,  H(t0,  K),V(t0,  K,  uext),V(t0,  K,  uext)^ 

k 

=  ^2  Mi  \xoHi(to,  K)x0 

i=l 

+  2xoVi(t0,  K,  uext)  +  Vi  (to ,  K,  uext )]  (26) 

where  the  scalar,  real  constants  /ij  represent  parametric  de¬ 
sign  freedom  and  the  unique  solutions  {Hi(to,K)  >  0}^-, , 

\v>i(to,  K,  uext)y  _  and  {Vi(t0,  K,  uext)}k=1  evaluated  at 
a  =  to  satisfy  the  dynamic  equations 

— H(a)  =  T{a,  H(a),K{a)),  H(tr)  =  Hr, 
da 

^Vlya)  =  Q(a,  H(a),V(ya),K(a),uext{a^J ,  V(tf)  =  Vf, 

-^-V(a)  =  Q  (a,H(a),V(a),uext(a)^  ,  V{tf)  =  Vf  . 

For  given  terminal  data  (tf,Hf,Vf,Vf),  the  classes  of 
admissible  external  signal  and  feedback  gains  may  be  defined 
as  follows. 

Definition  2:  (Admissible  Signal  and  Feedback  Gains) 
Let  compact  subsets  U  C  Rm  and  K  C  Rmxn  be  the  sets 
of  allowable  external  inputs  and  gain  values.  For  the  given 
k  £  Z+  and  the  sequence  fi  =  {//./  >  0}^=1  with  fi x  > 
0,  the  set  of  admissible  external  signals  U.  ^  ^  and 
feedback  gains  ^  t>  -fi  316  respectively  assumed  to 

k  be  the  classes  of  C([Ao,  f/];  Km)  and  C([to,  tf]\ Kmxn)  with 
values  uext(-)  £  U  and  K(-)  £  K  for  which  solutions  to  the 
dynamic  equations  with  the  terminal  conditions  H(tf)  = 
V(tf)  =  Vf,  and  V(tf)  =  Vf 

^H(a)  =  F(a,H(a),K(a)),  (27) 

da 

-j^'D(a)  =  Q  (a,H(a),V(a),  K (a) ,  uext(a2)  ,  (28) 

=  Q  (a,H{a),V(a),uext(a)^)  (29) 

exist  on  the  interval  of  optimization  [f0,A/]- 
Then  the  optimization  statements  for  the  state-feedback  A  CC 
control  of  the  tracking  problem  over  a  finite  horizon  may  be 
stated  in  the  sequel. 

Definition  3:  (Optimization  Problem) 

Suppose  that  k  £  Z+  and  the  sequence  /r  =  {/ij  > 
0}?=i  with  fix  >  0  are  fixed.  Then  the  state-feedback 
fcCC  control  optimization  problem  over  [A(J.  t  f  is  given  by 
the  minimization  of  (26)  over  uext{-)  £ 

K(-)  £  K,t  f  H  d  ft  and  subject  to  the  dynamic  equations 
of  motion  (27)-(29)  for  a  £  [to,tf]. 

The  sequence  of  following  results  will  discuss  the  construc¬ 
tion  of  scalar-valued  functions  which  are  the  candidates  for 
the  value  function. 

Definition  4:  (Reachable  Set) 

Let  reachable  set  Q  be  defined  Q  =  j  Z,zJ  £ 


[to,  tf  ]  x  (Rnx")fc  x  (Kn)fc  x  Kfe|  such  that  U£  y  ±  z.^  ±  0 
and  ^Zy^z,z-,!x  ^ 

By  adapting  to  the  initial  cost  problem  and  the  termi¬ 
nologies  present  in  the  kCC  control,  the  Hamilton-Jacobi- 
Bellman  (HJB)  equation  satisfied  by  the  value  function 
V  (e,y,Z,z)  is  then  given  as  follows. 

Theorem  3:  (HJB  Equation-Mayer  Problem) 

Let  (e,y,Z,z)  be  any  interior  point  of  the  reachable  set  Q 

at  which  the  value  function  V  (e,y,Z,z)  is  differentiable. 
If  there  exist  optimal  external  signal  u*xt  £Me  y  jt  Zl,  and 
feedback  gain  K*  £  y  z  z.^  then  the  partial  differential 
equation  of  dynamic  programming 


0  = 


xteu,KeK  I  de 


d 

dvec(y) 
d 


V 


p,y,z,z) 


(30) 


V  (e,  y,  Z,  z)  vec  (tF  (e,  y,  K)) 


+  - - PY  v  (£j  y>  z) vec  ip  (£’ 371  Uext) ) 


d  vec  (z) 
d 


3  vec  (Z) 


v(e,;y,z,z)vec  (g  (e,y,Z,uext 


is  satisfied  wherein  the  boundary  condition 
V  (to,  Ho,  V0,  Vo)  =  (j)tk  (to,  Ho,  V0,  V0) . 

Proof.  Refer  to  the  reference  [7]  for  the  detailed  proof. 

Theorem  4:  (Verification  Theorem) 

Fix  k  £  Z+  and  let  W  (e,y,Z,z)  be  a  continuously  dif¬ 
ferentiable  solution  of  the  HJB  equation  (30)  which  satisfies 
the  boundary  condition 

W  (to, Ho,  Vo,  Vo)  =<tHu  (to, Ho,  Vo,  Vo)  .  (31) 

Let  (tf,  Hf,  V  f ,  Vf)  be  in  Q;  ( uext ,  K )  in  x 

JCt  £  D  .y,  H,  V  and  V  the  corresponding  solutions 
of  (27)-(29).  Then  W(a,H(a),V(a),V(a))  is  a  non¬ 
increasing  function  of  a.  If  ( u*xt ,  K*)  is  in  Utf  n  ^  v  x 
JC  Hf  $  D  ,/(  defined  on  [to,tf]  with  corresponding  solu¬ 
tions,  TL* ,  V* ,  and  V*  of  (27)-(29)  such  that  for  a  £  [to,tf] 


0  =  ^-W  (a,H*(a),V*(a),V*(a)) 

+  d^y)w^n*ia)^{a)^*{a))' 

•vec  (Z(a,H*(a),K*(a))) 

+ - (a,H*(a),f>*(a),V*(a)]  ■ 

■  vec  (g  (a,H*(a),  V*  (a) ,  K*  (a) ,  u*ext  (a))  ) 

+  dJyz)W  (a,H*(a),V*(a),V*(a))  ■ 

■  vec  (p  (a,H*(a),T>*(a),u*xt(a)))  ,  (32) 


then  Kxt  and  K*  are  optimal.  Moreover, 

w(e,y,z,z)  =  v(e,y,z,z)  (33) 

where  V  (e,y,Z,z)  is  the  value  function. 

Proof.  Refer  to  [7], 

III.  Optimal  Solution  of  kCC  Control 

The  treatment  of  HJB  approach  to  obtaining  a  state- 
feedback  solution  to  the  fcCC  control  problem  over  the  finite 
horizon  of  optimization  requires  to  parameterize  the  terminal 
time  and  states  of  the  dynamical  equations  as  (e,y,Z,z) 

rather  than  (tf,Hf,Vf,Vf).  That  is,  for  e  £  [to,tf]  and 
1  <  i  <  k,  the  states  of  the  system  (27)-(29)  defined  on  the 
interval  [to ,  e]  have  the  terminal  values  denoted  by 

H(e)  =  y ,  V(e)  =  Z ,  V(e)  =  Z  . 

Observe  that  the  performance  index  (26)  is  quadratic  affine 
in  terms  of  the  arbitrarily  fixed  xo-  This  suggests  a  solution 
to  the  HJB  equation  (30)  may  be  sought  in  the  form 

k 

w  (e,  y,  z,  z)  =  4  ^  m  (yt  +  £&))  x0 

2=1 

k  k 

+  2xq  ^2  m  (j^i  +  %(£))  +  ’22  Hi  ( Zj  +  Ti(e))  (34) 

i=l  i= 1 

where  these  parametric  functions  of  time  £)  £ 

(^1([to,tf];Sn),  %  £  C1([f0,f/];Rn)  and  7^  £  C1([f0,f/];K) 
are  to  be  determined.  Using  the  isomorphic  vec  mapping, 
there  is  no  difficulty  to  verify  the  following  result. 

Lemma  1:  Fix  k  £  Z+  and  let  (e,  y,  Z,  z)  be  any 
interior  point  of  the  reachable  set  Q  at  which  the  real-valued 
function  W  (e,y,Z,z)  of  the  form  (34)  is  differentiable. 

The  derivative  of  >V  (s,y,  Z,  z)  with  respect  to  £  is  given 


d_ 

de 


w(e,y,Z,z)  =  xlY,^i{^y,K)+  (e)) 

k  /  d  \ 

+  2*0  ^2  Vi  (pi  (e>  Z,  K,  uext)  +  —71(e)) 


i— 1 
k 


+ 


yGi  (y,y,Z,uext)  +  ygV^e)  )  ,  (35) 


provided  uext  €  U  and  K  £  K. 

Trying  the  guess  solution  (34)  into  the  HJB  equation  (30),  it 
follows  that 


(36) 


Notice  that 


+  ^  myiB(e)R~1(e)BT(e)  Y  £rir 

2=  1  r=  1 

k  k 

-Hi  Y  Mr  yrB(e)R-1(e)BT(e)  Y 


+  Y  -  Y  /J'Tr  {^:G(^GT(e)} 

2=1  2=1 
k  k 

+2Y,HiZjB{£)R~1{e)BT{e)  Y  HrZr- HizT(e)Q(£)z(£) 

i—  1  r— 1 

fe  k 

-  Y  VrZr  B(£)R~1(e)BT(e)  Y  HsA .  (39) 


The  remaining  task  is  to  display  time-dependent  functions 

{£»(■)}*=  i>  {^(')}._  >  and  {^i(')}i=i>  which  yield  a  suf¬ 
ficient  condition  to  have  the  left-hand  side  of  (39)  being 

zero  for  any  e  £  [to,t/],  when  {34}i=i  and  are 

evaluated  along  solutions  to  the  cumulant-generating  equa- 
tions.  Careful  observation  of  (39)  suggests  that  {£*(■) }»=i> 

{^(•)V  and  {7j (•)}*_  ,  may  be  chosen  to  satisfy  the 
differentia]1  equations  as  follows 


~r£  i(e)  =  AT(e)TL  i(e)  +  H\{e)A{e)  +  Ct(£)Q(e)C(£) 
as 

k 

-  Hi{£)B(e)R~1(£)BT(£)  Y  TlsBLs{£) 

8=  1 
k 

-  Y/Hr'Hr{£)B(£)R~1(£)BT(£)Hl(£) 

r—1 

k  k 

+  YHrnr{£)B{£)R-1{£)BT{£)YHsHs{e) ,  (40) 

r—1  s—1 

=  AT  {s)T~Li{£)  +  Hi{e)A{e) 
as 

k 

-  Hi(£)B(e)R~1(£)BT(£)  ^2  fisHsie) 

s= 1 
k 

-  firHr(£)B(£)R-1(£)BT(£)Hj(£) 

r—1 

l_1  2d 

+  E  jKi  1  j)[nj(£)G(£)wGT(£)Hi-j(e) ,  (41) 

1(e)  =  -4T(e)^t(£)  -  C^OOQOOzO?) 

d£ 

k 

-  Y ]  Hr'Hr(£)B{£)R~1  {£)BT (£)V i(g) 


+2^0  YaijZ %(£)-AT(£)YAiZi+HiCT(£)Q(£)z{£)  -Hi{£)B{£)R  1(e)BT(e)  y^j5rZ?r(e) 

_  2—1  2=1  r=l 

fc  k  k  k 

+  X!  HryrB(e)R~1(e)BT(e)  ^  /x»-Zi  +  E  Hr'Hr(e)B(£)R~1(£)BT(£)  ^  Msl>s(£) 


Y  y ,  K)  =  -  [A(s)  +  B{e)K] T  Y  H^i 

2=1  2=1 

k 

-  Y  MiX  Me)  +  -  /xtC^eJQ^Cle) 

2=1 

i 

-^KtR(£)K  -Y^Y  u2-  ■vyjG(e)wGF'(e)yi-j, 

i= 2  j=i 

k  k 

Y  hA  (e,  y,  2,  K,  uext )  =  -  [A(e)  +  B(e)  Af  ’  ^  ^ 

2=1  2=1 

k 

-  Y  ^y*B(£)uext  -  m KTR{£)uext  +  n1CT(e)Q(e)z(e), 

k  k 

Y^iGi  {e,y,2,uext)  =  -Y^{y^G^WGT(£)} 
2=1  2=1 

k 

-2^  HiZ?B(£)uext-  Lm£xtR(£)uext- mzT(£)Q(£)z(£). 

i= 1 

Since  the  initial  condition  ,t0  and  A/(J  are  arbitrary  vector  and 
rank-one  matrix,  the  necessary  condition  for  an  extremum  of 
(26)  on  [2q  ,  S']  is  obtained  by  differentiating  the  expression 
within  the  bracket  of  (36)  with  respect  to  uext  and  K  as 

k 

Uext{£,  Z)  =  -R~1{£)BT(£)  Y  HrA  ,  (37) 

r=l 

k 

K(£,  y)  =  -R~1(e)Bt(£)  Y  Hryr  ,  (38) 

r=l 

where  /xr  =  fJLi/ n  1  and  fi i  >  0.  Replacing  (37)  and  (38)  into 
(36)  leads  to  the  value  function 


E  -  At(£)  Y  H&i  ~  E 

,i=l  £  i= 1  i—1 

Hi  CT{£)Q{£)C{£) 

k  k 

+  Y  HryrB{£)R-1(£)BT(£)  Y  HM 

r—1  2=1 

k  k 

+  Y  HiyA)B(e)R~1(s)BT(e)  Y  Ms34 

2=1  3=  1 

k  k 

-  Ll1YHryrB{£)R~1{£)BT(£)Y'P'sys 


-  E  *  E  w—yy1G{£)WGT{£)yl-J j  z0 


(42) 


=  AT(£)Vi(£) 
as 

k 

-YJ»rHr{£)B{£)R-1{£)BT{e)Vi{e) 

r—1 

k 

-  Hl(£)B{£)R-\£)BT{£)J2$rVr(£) ,  (43) 

r—1 

^Ti(e)  =  Tx{H1{£)G{£)WGt{£)}  +  zt(£)Q(£)z(£) 

k 

-  Wf  ( £)B(£)R-\£)BT{£ )  ^r^r(e) 

r—1 

k  k 

+  YJVrVl(£)B(£)R-\£)BT(£)YJ^sVs{£)  ,  (44) 

r—1  s—1 

*-%{£)  =  Tr  {'Hi(e)G(e)WGT  (e)} 

k 

-  2 iff  ( £)B{£)R-1{£)Bt{£ )  Y  ArA{£)  ■  (45) 

r=l 

The  external  signal  and  feedback  gain  specified  in  (37)  and 
(38)  are  now  applied  along  the  solution  trajectories  of  the 
equations  (27)-(29) 

-y-Wi(e)  =  ~AT {e)H\(e)  —H\{e)A{e)  -  CT (e)Q(e)C(e) 
as 

k 

+  H1(£)B(£)R~1(£)BT (£)  Y  V's'Hs (£) 

S  —  1 

k 

+  ^2  Pr'Hr{£)B(£)R~1(£)BT (£)Hi(e) 

r—1 

k  k 

-  J2  %-Hr{e)B{£)R-\£)BT{£)  Y  Wsie) ,  (46) 

r=l  s=  1 

=  -AT(£)Hi(£)  -  Hi(e)A(e) 
as 

k 

+  ni(£)B(£)R-1(£)BT(£)  Y  ns{£ ) 

S—1 

k 

+  Yj  P‘r'Hr{£)B(£)R~1(£)BT(£)Hl(£) 

r—1 

1-1  9  »'l 

-  E  j^—ynj{e)G{e)WGT{e)'Hi_j{e) ,  (47) 

=  -AT(£)V1(£)  +  CT  {£)Q{£)z{e) 
as 

k 

+  Y  TwHr{e)B{e)R-1{e)BT{e)V1{e) 

r=  1 

k 

+  Hi(£)B(£)R~l(£)BT (£)  Y,  V>r'Dr{e) 

r—1 

k  k 

-  Y  MrWr(e)B(e)iZ"1(e)Br(e)  Y  ZMe)  ,  (48) 

r—1  s=l 


7  ^ 

=  Y  ^rUr{£)B{£)R-1{£)BT{£)Vi{£)  (49) 

r—1 

k 

-  AT{£)Vi(e)  +Hi{£)B{£)R~1{e)BT{e)  y^j5rPr(e) , 

r=l 

=  -Tr{7fi(£)G(£)W/GT(£)}  -  zT(£)Q(£)2:(£) 

k 

+  2F>f  (£)B(£)i2-1(£)BT(£)  ^  JirAis) 

r=  1 

k  k 

-  Y  fc&T  (e)B(£)R~1(e)BT(£)  Y  &#a(<0 ,  (50) 

r—1  s=l 

=  -Tr  {Ht{£)G{£)WGT{£)} 

k 

+  2 Vj(e)B(e)R-1{e)BT(e)Y'iirVr{E)  ■  (51) 

r=l 

where  the  terminal  conditions  7fi (tf)  =  CT(tf)QfC(tf), 
=  0  for  2  <  i  <  fc;  T>i(tf)  =  — CT(tf)Qfz(tf ), 
=  0  for  2  <  i  <  k  and  T>i(tf)  =  zT(tf)Qfz(tf), 
=  0  for  2  <  i  <  k.  The  boundary  condition  of 
W(£,y,Z,Z)  implies  that 

k 

xo  Y  ^  (^*°  +  ^(*o)) 

*=1 

k  k 

+  (ViO  +  ^  (£()))  +  Mi  (^iO  +  (to)) 

i=l  i— 1 

fc  k  k 

—  Xq  Y^j  HiDiQ  +  HiDiQ  . 

i— 1  i=l  i=l 

The  initial  conditions  for  the  equations  (40)-(45)  follow 
£i(to)  =  0,  Ti(to)  =  0,  and  %(to)  =  0.  Therefore,  the 
optimal  external  signal  (37)  and  state-feedback  feedback  gain 
(38)  minimizing  the  performance  index  (26)  become 

k 

Kxt(£ )  =  -#_1(e)£T(£)E^*(£)  ’ 

r=l 

k 

K*{£)  =  -R-\£)BT{£)  Y  $tK(£)  ■ 

r—1 

The  theorem  that  follows  contains  a  controller  design  algo¬ 
rithm  which  is  able  to  track  a  prescribed  function  of  time  in 
the  optimal  fcCC  sense.  The  optimal  kCC  tracking  controller 
requires  a  standard  state-feedback  fcCC  control  design  and 
an  additional  signal  that  results  from  the  backward  solutions 
of  linear  differential  equations. 

Theorem  5:  (Finite-Horizon  fcCC  Control  Solution  for 
Tracking  Problems) 

Let  A  e  C([f0,f/];Rnxn),  B  e  C([f0,*/];Rnxm),  G  e 
C([io)  tf];  Krxn),  and  G  G  C([fo,  tf};  Knxp).  The  tracking 
problem  is  then  described  by  the  equations  (l)-(2)  where 
the  input  noise  w(t)  £  1RP  is  the  p-dimensional  Wiener 
process  starting  from  to,  independent  of  the  initial  con¬ 
dition  xq,  and  defined  on  a  complete  probability  space 


(n,!F,V)  over  [i0,i/]  with  the  correlation  of  increments 
E  {[w(t)  —  tu(£)][ti>(r)  —  U,(£)]T}  =  W\t  —  £|  and  W  > 
0.  The  control  input  u  £  Lpt(Q]C([to,tf];Wn))  to  the 
specified  system  is  selected  so  that  the  resulting  output 
y  £  L^t  (Q;  C([foi  t/];  Mr))  best  approximates  the  known  a 
priori  trajectory  z  £  L2(C([to,  tf]\ Rr))  in  the  sense  of  (26) 
in  which  the  terminal  penalty  error  weighting  Qf  £  Kr’xr, 
the  error  weighting  Q  £  C([to,  f/];  Rr’xr),  and  the  control 
input  weighting  R  £  C([to,  f/];  Rmxm)  are  symmetric  and 
positive  semidefinite  with  R(t)  invertible. 

Assume  both  k  £  Z+  and  the  sequence  y  =  {yi  >  0},f=1 
with  yi  >  0  are  fixed.  Then,  the  linear  state-feedback  fcCC 
control  solution  for  the  finite-horizon  tracking  problem  is 
implemented  by 

u*(t)  =  K*(t)x*(t)  +  u*ext(t),  (52) 

k 

K*(a )  =  -R-1(a)BT(a)^2yrH*(a) ,  (53) 

r= 1 
k 

u*ext(a)  =  -R-1(a)BT(a)J2$rV:(a) ,  (54) 

r—1 

where  yr  =  yi/yi  and  whenever  {W*(a)}r=1,  and 

I'D* (a)  |  are  the  solutions  of  the  backward-in-time  ma¬ 
trix  differential  equations 

-f- HUa )  =  -  \A{a)  +  B(a)K*(a)]T  HU  a)  (55) 

da 

-  HI  (a)  [A(a)  +  B(a)K*(a)} 

-  CT (a)Q(a)C(a)  -  K*T {a) R{a) K* (a) , 

4- H*(a )  =  -  [A{a)  +  B(a)K*(a)]T  H*(a)  (56) 

da 

-  H*(a)  [ A(a )  +  B{a)K*(a)} 

Dll  or' 

-  E  M-  -  ^K{<x)G{a)WGT{a)U*r_s{a), 

SAT  S). 

s— 1 

and  the  backward-in-time  vector  differential  equations 

=  -  [A(a)  +  B(a)K*{a)]T  VHa)  (57) 

da 

-  Hi(a)B(a)u*ext(a) 

—  I<*T (a)R(a)u*ext(a)  +  CT (a)Q(a)z(a) , 

E©;(a)  =  -  [A{a)  +  B{a)K*{a)]T  V*r{a) 

-Hr(a)B(a)u*ext(a)  (58) 

with  the  terminal  boundary  conditions  Til  (tf)  = 
CT(tf)QfC(tf),  H*(tf)  =  0  for  2  <  r  <  k  and 
V*(tf)  =  —CT(tf)Qfz(tf),  V*(tf)  ~  0  f°r  2  <  r  <  k. 

IV.  Conclusions 

In  this  paper,  an  optimal  control  problem  for  a  wide  class 
of  tracking  systems  is  formulated  in  which  the  objective  is 
minimization  of  a  finite,  linear  combination  of  cumulants 
of  integral  quadratic  cost  over  linear,  memoryless,  full-state- 
feedback  control  laws.  The  standard  linear  tracking  system 
constraint  on  a  finite  time  interval  with  additive  Wiener 


noise  and  a  non-random  initial  state  underlies  the  problem 
formulation.  Because  of  the  linearity  assumptions  in  the 
problem  statement,  it  can  be  formulated  as  a  non-stochastic 
optimization  problem  utilizing  equations  for  cost  cumulants 
developed  in  this  exposition.  Furthermore,  this  problem  for¬ 
mulation  is  parameterized  both  by  the  number  of  cumulants 
and  by  the  scalar  coefficients  in  the  linear  combination,  it 
defines  a  very  general  Linear-Quadratic-Gaussian  (LQG)  and 
Risk  Sensitive  problem  classes.  The  special  cases  where  only 
the  first  cost  cumulant  is  minimized  is,  of  course,  the  well 
known  minimum  mean  LQG  problem  and  whereas  a  denu¬ 
merable  linear  combination  of  cost  cumulants  is  minimized  is 
the  continued  Risk  Sensitive  control  objective.  It  should  also 
be  noted  that  although  the  optimization  criterion  of  the  cost- 
cumulant  control  problem  represents  a  competition  among 
cumulant  values,  the  ultimate  objective  herein  is  to  introduce 
parametric  freedom  in  the  class  of  feedback  control  laws 
which  will  result  from  the  problem  solution.  This  parametric 
freedom  has  been  exploited  to  achieve  desirable  closed- 
loop  system  properties  as  illustrated  in  [l]-[5].  Finally,  the 
general  solution  of  the  cost-cumulant  control  problem  for  the 
class  of  linear-quadratic  tracking  systems  is  presented  and 
is  determined  by  a  feedback  cost-cumulant  control  obtained 
by  a  set  of  coupled  Riccati-type  differential  equations  and 
time-dependent  tracking  variables  found  by  solving  an  aux¬ 
iliary  set  of  coupled  differential  equations  (incorporating  the 
desired  trajectory)  backward  from  a  stable  final  time.  The 
issue  of  existence  of  solution  to  the  optimization  problem 
becomes  that  of  existence  of  solutions  to  the  Riccati-type 
equations.  Conditions  ensuring  existence  of  solutions  to 
these  equations  are  being  worked  out.  In  fact,  for  values  of 
linear  combination  coefficients  outside  certain  finite  ranges, 
the  equations  exhibit  finite  escape  time  behavior.  On  the 
other  hand,  for  limited  ranges  of  the  combination  coefficient 
values,  the  equations  are  well  behaved  and  yield  steady-state 
solutions  as  shown  in  various  controller  designs  [l]-[5]. 
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